Bayesian Optimisation (BO) refers to a class of methods for global optimisation of a function f which is only accessible via point evaluations. It is typically used in settings where f is expensive to evaluate. A common use case for BO in machine learning is model selection, where it is not possible to analytically model the generalisation performance of a statistical model, and we resort to noisy and expensive training and validation procedures to choose the best model. Conventional BO methods have focused on Euclidean and categorical domains, which, in the context of model selection, only permits tuning scalar hyper-parameters of machine learning algorithms. However, with the surge of interest in deep learning, there is an increasing demand to tune neural network architectures. In this work, we develop NASBOT, a Gaussian process based BO framework for neural architecture search. To accomplish this, we develop a distance metric in the space of neural network architectures which can be computed efficiently via an optimal transport program. This distance might be of independent interest to the deep learning community as it may find applications outside of BO. We demonstrate that NASBOT outperforms other alternatives for architecture search in several cross validation based model selection tasks on multi-layer perceptrons and convolutional neural networks.

Automatic neural architecture design has shown its potential in discovering powerful neural network architectures. Existing methods, no matter based on reinforcement learning or evolutionary algorithms (EA), conduct architecture search in a discrete space, which is highly inefficient. In this paper, we propose a simple and efficient method to automatic neural architecture design based on continuous optimization. We call this new approach neural architecture optimization (NAO). There are three key components in our proposed approach: (1) An encoder embeds/maps neural network architectures into a continuous space.

In this paper we propose a generalization of this work that generally exhibits improved performace, but from an implementation point of view is actually simpler.

Such a better embedding is then decoded to a network by the decoder.

This paper presents a text classification algorithm inspired by the notion of superposition of states in quantum physics.

The Job Shop Scheduling (JSS) problem can be viewed as an integer optimization program with linear objective function and linear, disjunctive constraints. Theconstraints(14c)enforceprecedencebetween tasks that must be scheduled in the specified order within their respective job. Themodel presented belowisusedtoconstruct solutions that are integral, and feasible tothe original problem constraints. However, the resolution frequency to solve OPFs is limited by their computational complexity. Additionally,the stochasticity introduced by renewable energy sources further increases the number of scenarios to consider. C.2 DatasetDetails Table 4 describes the power network benchmarks used, including the number of buses|N|, and transmission lines/transformers |E|.

We call this family of functions intra order-preserving functions. We propose a new neural network architecture that represents a class of intra order-preserving functions by combining common neural network components.